Part of the game: Changing level creation to identify and filter low quality user-generated levels

While there are many potential benefits of user-generated content for serious games, the variability of that content’s quality poses a serious problem. In our game, BOTS, players can create puzzles which are shared with other users. However, other players often find these puzzles irrelevant, unplayable, too difficult, or simply boring. To avoid frustrating players with low-quality puzzles, we have implemented a “Solve and Submit” process, where a player must “set the bar” for their level by providing a solution. We compare the levels that make it through this submission process to levels created when there is no such mechanism in place. We show that employing a self-evaluation mechanic for user generated content will reduce the number of low-quality puzzles submitted.